Skip to content

Remove the superseded v5/v6/v7 search schema revisions - #3523

Open
jonathangreen wants to merge 2 commits into
mainfrom
chore/remove-old-search-schema-revisions
Open

Remove the superseded v5/v6/v7 search schema revisions#3523
jonathangreen wants to merge 2 commits into
mainfrom
chore/remove-old-search-schema-revisions

Conversation

@jonathangreen

@jonathangreen jonathangreen commented Jun 30, 2026

Copy link
Copy Markdown
Member

Description

Stacked on top of #3522 (feature/search-v8). Removes the superseded v5/v6/v7 search schema revisions now that v8 is self-contained and is the only revision production creates indexes from.

  • Deletes src/palace/manager/search/v5.py, v6.py, and v7.py, along with their dedicated tests (test_search_v6.py, test_search_v7.py).
  • Drops the old revisions from SearchRevisionDirectory, leaving REVISIONS = [SearchV8()].
  • Repoints the remaining references at v8: the MockSearchSchemaRevisionLatest mapping document and a name_for_index docstring example.
  • Moves the author char-filter test (previously written against v5 in test_external_search.py) into TestSearchV8, and drops v8's v7-equivalence test now that v7 no longer exists to compare against.

Motivation and Context

#3522 made v8 self-contained specifically so that older revisions could be deleted once they fell out of use, instead of accumulating indefinitely behind the inheritance chain. This PR is that cleanup. Opening it now (in draft) keeps the follow-up visible so the removal isn't forgotten after the v8 release goes out.

How Has This Been Tested?

Ran the affected suites under the docker tox environment — tests/manager/search/, tests/manager/scripts/test_initialization.py, and tests/manager/celery/tasks/test_search.py (the latter two exercise MockSearchSchemaRevisionLatest, whose backing document now comes from v8). All 171 tests pass. mypy is clean on the changed source.

Checklist

  • I have updated the documentation accordingly.
  • All new and existing tests passed.

@claude

claude Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Claude finished @jonathangreen's task in 3m 29s —— View job


Code Review

  • Gather context (read PR diff and changed files)
  • Check for dangling references to v5/v6/v7
  • Verify test migration fidelity
  • Review revision directory / mock changes
  • Post review

Summary

Clean, mechanical removal. I verified there are no remaining Python references to SearchV5/SearchV6/SearchV7 anywhere in src/ or tests/, that SearchV8 is genuinely self-contained (no imports from the deleted modules, AUTHOR_CHAR_FILTER_NAMES/CHAR_FILTERS are defined on SearchV8 itself so the migrated test binds correctly), and that the only runtime consumers of the directory — celery/tasks/search.py:120 and scripts/initialization.py:140 — call highest(), so trimming REVISIONS to a single entry doesn't change any lookup path. The char-filter test is a faithful port (identical assertions, just repointed at SearchV8), and import re moved with it. The release-ordering guard in the description is the right call per CLAUDE.md's online-migration rule. Two minor items below; neither blocks.

Details

Minor: docker/Dockerfile.os2.ci:5

Both CI OpenSearch images justify installing analysis-icu with a pointer to src/palace/manager/search/v5.py, which this PR deletes — the reference is now dangling. v8.py is what needs the plugin (icu_collation_keyword at v8.py:224 and the icu_collation filter at v8.py:198), so repointing both comments at v8.py keeps them useful:

# analysis-icu is required by the search schema (see src/palace/manager/search/v8.py).

# analysis-icu is required by the search schema (see src/palace/manager/search/v5.py).

# by the search schema (see src/palace/manager/search/v5.py).

Minor: tests/manager/search/test_search_v8.py:29-39 (removed)

Dropping test_mapping_matches_v7 is right (there's nothing left to compare against), but it was the only test pinning v8's mapping structure — what remains asserts the version, the base class, the index settings keys, and the char filters, none of which cover the field set or analysis config. Since revisions are documented as immutable once created (revision.py:5-11), a stray edit to v8.py after release would silently diverge from production indexes without a version bump, and only the docker end-to-end suite would have any chance of catching it. A cheap replacement guard is a snapshot assertion over the serialized document:

def test_mapping_is_stable(self):
    """v8 is released; its mapping must not change without a new revision."""
    document = SearchV8().mapping_document()
    assert set(document.properties) == {"summary", "title", ...}
    assert document.serialize_properties() == EXPECTED_V8_PROPERTIES

"""v8 is a faithful, self-contained copy of the v7 schema: same fields
and same analysis settings. Only the index settings differ (v8 pins the
shard count)."""
v8_document = SearchV8().mapping_document()
v7_document = SearchV7().mapping_document()
assert v8_document.serialize_properties() == v7_document.serialize_properties()
assert v8_document.settings["analysis"] == v7_document.settings["analysis"]
# v7 left the shard count to an inherited default; v8 sets it explicitly.
assert "index" not in v7_document.settings
assert "index" in v8_document.settings


Note: I couldn't execute the test suite in this environment (sandbox blocked pytest), so my review is static analysis of the diff plus reading the affected call sites; CI and Codecov on this PR are green.
| Branch: chore/remove-old-search-schema-revisions

@greptile-apps

greptile-apps Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR removes the now-superseded v5, v6, and v7 search schema revision modules, their tests, and all references to them, leaving REVISIONS = [SearchV8()] as the sole entry in SearchRevisionDirectory. It is the companion cleanup to #3522, which made v8 self-contained.

  • Deletes v5.py, v6.py, v7.py and their dedicated test files; removes the old entries from REVISIONS.
  • Ports the author char-filter test from the deleted TestSearchV5 into TestSearchV8, and drops the now-unnecessary v7-equivalence test.
  • Updates MockSearchSchemaRevisionLatest and the name_for_index docstring to reference v8.

Confidence Score: 5/5

Safe to merge — this is a straightforward dead-code removal with no functional changes to production paths.

All v5/v6/v7 references have been scrubbed from the codebase (confirmed via grep), v8 is self-contained and unchanged functionally, tests were ported correctly, and the mock backing document now matches the current production schema.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
src/palace/manager/search/revision_directory.py Drops v5/v6/v7 imports and narrows REVISIONS to [SearchV8()]; clean and correct.
src/palace/manager/search/v8.py Docstring trimmed to remove references to the old inheritance chain; no functional changes.
tests/mocks/search.py MockSearchSchemaRevisionLatest now backed by SearchV8().mapping_document() — correct for the current production schema.
tests/manager/search/test_search_v8.py Removes v7-equivalence test (v7 deleted) and adds test_character_filters ported verbatim from the old TestSearchV5, now targeting SearchV8 — correct.
tests/manager/search/test_external_search.py Drops the TestSearchV5 class and its v5/re imports; no remaining references to deleted modules.
src/palace/manager/search/revision.py One-line docstring update: example index name updated from v5 to v8.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    RD["SearchRevisionDirectory\n(revision_directory.py)"]
    V8["SearchV8\n(v8.py)"]
    SR["SearchSchemaRevision\n(revision.py)"]

    RD -->|"REVISIONS = [SearchV8()]"| V8
    V8 -->|inherits| SR

    subgraph deleted["Deleted (this PR)"]
        V5["SearchV5 (v5.py)"]
        V6["SearchV6 (v6.py)"]
        V7["SearchV7 (v7.py)"]
        V6 -->|"was: inherits"| V5
        V7 -->|"was: inherits"| V6
    end

    subgraph mocks["Test Mocks"]
        MSSL["MockSearchSchemaRevisionLatest"]
        MSSL -->|"_document = SearchV8().mapping_document()"| V8
    end

    style deleted fill:#ffdddd,stroke:#cc0000,color:#333
Loading

Reviews (3): Last reviewed commit: "Trim stale revision-chain history from t..." | Re-trigger Greptile

@codecov

codecov Bot commented Jun 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.50%. Comparing base (53e3a1d) to head (2b54d93).
⚠️ Report is 31 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3523      +/-   ##
==========================================
- Coverage   93.52%   93.50%   -0.02%     
==========================================
  Files         512      509       -3     
  Lines       46760    46663      -97     
  Branches     6379     6378       -1     
==========================================
- Hits        43731    43633      -98     
- Misses       1958     1959       +1     
  Partials     1071     1071              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jonathangreen
jonathangreen requested a review from a team June 30, 2026 14:26
@jonathangreen

Copy link
Copy Markdown
Member Author

I'm going to leave this in draft until its safe to merge, but its ready for a code review now, then it can just wait to get merged once approved.

@tdilauro tdilauro left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! 🏁

Base automatically changed from feature/search-v8 to main July 7, 2026 14:44
v8 is self-contained and is the only revision production creates indexes
from, so the older chained revisions (v7 -> v6 -> v5) are no longer
referenced and can be deleted. This removes their modules, drops them from
the revision directory, and repoints the remaining test/mock usages at v8.

The v8 module test absorbs the author char-filter test that previously
lived against v5 in test_external_search, and drops its v7-equivalence
test now that v7 is gone.
With v5/v6/v7 removed in this PR, the docstring no longer needs to recount
the old inheritance chain to justify v8 being self-contained.
@jonathangreen
jonathangreen force-pushed the chore/remove-old-search-schema-revisions branch from 1558880 to 2b54d93 Compare July 31, 2026 13:57
@jonathangreen
jonathangreen marked this pull request as ready for review July 31, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants